1. Field of the Invention
The present invention relates to a three-dimensional measurement apparatus, a measurement method therefor, and a computer-readable storage medium.
2. Description of the Related Art
In recent years, it is becoming a common practice to perform complex tasks using robots in place of humans. A typical example of such complex tasks is assembly of industrial products. For example, to grip a component by an end effector such as a hand of a robot, it is necessary to measure the relative position and orientation between the component and the robot (hand), work out a movement plan based on the obtained measurement result, and control an actuator. Such position and orientation measurement is necessary not only in gripping a component by a robot, but also in various situations such as self-position estimation of the robot in its autonomous movement.
A two-dimensional image (density image/color image) obtained by a camera or a range image obtained by a range sensor is commonly used for the above-mentioned position and orientation measurement. “T. Drummond and R. Cipolla, “Real-time visual tracking of complex structures,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 7, pp. 932-946, 2002. (to be referred to as “Reference 1” hereinafter)” mentions a technique for measuring the position and orientation of an object by representing a three-dimensional shape model of the object using a set of line segments (wire-frame model), and fitting a projected image of three-dimensional line segments to edges detected on a two-dimensional image. Also, D. A. Simon, M. Hebert, and T. Kanade, “Real-time 3-D pose estimation using a high-speed range sensor,” Proc. 1994 IEEE International Conference on Robotics and Automation (ICRA '94), pp. 2235-2241, 1994. (to be referred to as “Reference 2” hereinafter) mentions a technique for measuring the position and orientation of an object by fitting a three-dimensional shape model (polygon model) of the object to a three-dimensional point cloud on the object surface obtained from a range image.
Known methods of acquiring a range image are passive measurement which requires no special illumination, and active measurement in which an object is irradiated in a special pattern for measurement. In general passive measurement, images of an object are sensed by two or more cameras, and the distance of each pixel is calculated in accordance with the triangulation principle based on associations between the respective images. In general active measurement, an object is irradiated with, for example, slit light or two-dimensional pattern light, and the distance of each pixel is measured in accordance with the triangulation principle. Another known active measurement is of the TOF (Time-of-Flight) scheme which uses the time taken for emitted light to travel to an object and be reflected back by the surface of the object.
Position and orientation measurement which uses a two-dimensional image is performed by minimizing the distance between an image feature and a projected image of a three-dimensional shape model “on a two-dimensional image”. For this reason, the measurement accuracy in this measurement method is not always high and is even low especially for a position component of a camera in the direction of depth. In contrast, position and orientation measurement which uses a range image is performed by minimizing the distance between a three-dimensional point cloud and a three-dimensional shape model “in a three-dimensional space”. For this reason, the measurement accuracy in this measurement method is higher than the method which uses a two-dimensional image, but the degree of freedom of a certain component cannot be measured with high accuracy if an object (e.g., a two-dimensional object) has a less distinctive shape.
Under the circumstances, it a demand exists to improve the measurement accuracy of the position and orientation by complementarily using a two-dimensional image and a range image. For example, since a large number of edges (image features) can be detected on a two-dimensional image in portions where changes in density are remarkable, such as the boundary between the background and the object and the positions of shifts between the planes, a two-dimensional image is desirably used in these portions. In contrast, since the distances can be stably measured in even portions where image features are hard to detect, a range image is desirably used in these portions.
In relation to these techniques, “Y. Hel-Or and M. Werman, “Pose estimation by fusing noisy data of different dimensions,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 17, no. 2, pp. 195-201, 1995. (to be referred to as “Reference 3” hereinafter)” discloses a method of measuring the position and orientation of an object by processing features on a two-dimensional image as three-dimensional points with indefinite depths. Also, “Hiura, Yamaguchi, Sato, and Inokuchi, “Real-time tracking of free-form objects by range and intensity image fusion,” The Transactions of the Institute of Electronics, Information and Communication Engineers D-II, vol. J80-DII, no. 11, pp. 2904-2911, 1997. (to be referred to as “Reference 4” hereinafter)” discloses a method of measuring the position and orientation using pieces of gradient information of a two-dimensional density image and range image without explicit feature detection from a two-dimensional image.
An operation for assembling industrial products is expected to speed up by using robots. For example, it is necessary to measure the position and orientation of an object moving on a belt conveyor, or to measure them while moving a measurement apparatus as placed on a robot arm.
A simple equipment configuration including, for example, a camera and projector needs to acquire, at different timings, an image sensed while an object is not irradiated in a special pattern, and that sensed while the object is irradiated in a distance measurement pattern. If the measurement object and the measurement apparatus stand relatively still, no problem occurs even if these two types of images are acquired at different timings.
However, if these two types of images are acquired at different timings while the measurement object or the measurement apparatus itself moves, the relative position relationship between the measurement object and the measurement apparatus varies between the time when the two-dimensional image is acquired and that when the range image is acquired. Therefore, in such a case, it is impossible to measure the position and orientation of an object with high accuracy.